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MyTeachingPartner-Math/Science ( MTP-MS ) is a system of two curricula (math and science) plus teacher 
supports designed to improve the quality of instructional interactions in pre-kindergarten classrooms 
and to scaffold children's development in mathematics and science. The program includes year-long 
curricula in these domains, and a teacher support system (web-based supports and in-person workshops) 
designed to foster high-quality curricular implementation. This study examined the impacts of the inter¬ 
vention on the development of mathematics and science skills of 444 children during pre-kindergarten, 
via school-level random assignment to two intervention conditions (Basic: MTP-M/S mathematics and 
science curricula, and Plus: MTP-M/S mathematics and science curricula plus related teacher support 
system) and a Business-As-Usual control condition (Ball). There were intervention effects for children's 
knowledge and skills in geometry and measurement as well as number sense and place value: Children 
in Plus classrooms made greater gains in geometry and measurement, compared with those in Ball class¬ 
rooms. Children in Plus classrooms also performed better on the number sense and place value assessment 
than did those in Basic or Ball classrooms. We describe the implications of these results for supporting 
the development of children’s knowledge and skills in early childhood and for developing and providing 
teachers with professional development to support these outcomes. 

© 2014 Elsevier Inc. All rights reserved. 


Introduction 

Early childhood experiences help develop the foundational 
mathematics and science skills that allow children to fully engage 
in creative problem solving, collaboration, and learning (National 
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Association for the Education of Young Children [ NAEYC] & National 
Council of Teachers of Mathematics [NCTM], 2002; National 
Research Council [NRC], 2006). In fact, early play and experiences 
that engage children with the real world can lead to significant 
informal mathematical and scientific understandings (Clements, 
2004a; Duschl, Schweingruber, & Shouse, 2007), and help develop 
the capacity for complex and abstract thought (Bowman, Donovan, 
& Burns, 2001). This informal knowledge provides the basis for 
the development of formal knowledge and skills across curricular 
domains, and particularly in mathematics and science (Bowman 
et al., 2001). Children’s early mathematics and science knowledge 
and skills predict later school achievement (Claessens & Engel, 
2013; Grissmer, Grimm, Aiyer, Murrah, & Steele, 2010; National 
Mathematics Advisory Panel [NMAP], 2008) and are a more signifi¬ 
cant predictor of later academic success than are early reading skills 
(Duncan et al., 2007). 

This potential is often not achieved, however, as opportuni¬ 
ties for learning are missed in the early childhood classroom with 
important mathematics and science concepts and skills covered in 
cursory ways (Balfanz, 1999), or as discrete topics without broad 
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linkages and applications (NAEYC & NCTM, 2002; NRC, 2005). Little 
instructional time is spent on mathematics and science activities. 
For example, results from the National Center for Early Develop¬ 
ment and Learning's (NCEDL) national studies (Early et al., 2010) 
showed that children were exposed to mathematics activities dur¬ 
ing only 8% of classroom time, and to science activities for only 11% 
of classroom time, compared to 17% of classroom time in Language 
and Literacy activities and most of the time observed spent in “no 
coded learning activity” (44%). 

Children from families with lower levels of education and 
income are at greatest risk for insufficient mathematics and sci¬ 
ence instruction; child care and early education programs which 
these children typically attend have been observed to be “of such 
low quality that learning and development are not enhanced and 
may even be jeopardized” (Bowman et al., 2001 , p. 8). Children from 
disadvantaged backgrounds demonstrate fewer key mathematical 
skills at school entry: Results from analyses using the Early Child¬ 
hood Longitudinal Study - Birth Cohort (ECLS-B) suggest that only 
45% of four-year-olds from very poor families are proficient with 
numbers and shapes, compared to 72% of peers from families at 
or above the poverty level (NCES, 2009). These children’s educa¬ 
tional experiences do not serve to close this gap; instead the gap has 
been observed to widen. Upon Kindergarten entry, children from 
families with two or more risk factors have shown a seven-point 
gap in mathematics performance compared to their more advan¬ 
taged peers; this gap has been seen to widen to 15 points by the 
conclusion of the third grade (NCES, 2011 ). Although high-quality 
learning experiences that build knowledge and skills are critical for 
all preschoolers, they are even more important for children from 
disadvantaged backgrounds (NMAP, 2008). 

High-quality curricula have been found to support children’s 
mathematics and science learning (Clements & Sarama, 2008; 
French, 2004; Starkey, Klein, & Wakeley, 2004). However, large- 
scale studies suggest that even when pre-k teachers are provided 
with validated curricula, they frequently struggle to implement 
them with quality and fidelity, likely due to teachers’ lack of 
subject-area content knowledge and confidence (Pianta et al., 
2005). This is a problem of particular importance in mathe¬ 
matics and science, where early childhood educators are not 
well-prepared, receive little professional development (NRC, 2006), 
and are less confident and less experienced than in other content 
areas (Copley, 2004; Stipek, 2008). Embedding professional devel¬ 
opment support within curricular materials can help encourage 
transfer of desired teaching practices to the classroom. Delivering 
this professional development support via the Internet can increase 
scalability and accessibility (NRC, 2007), and therefore may be more 
likely to make a detectible difference in the practice of a large num¬ 
ber of teachers. 

In this manuscript, we report the results of a year-long study 
designed to test the effects of the early childhood mathematics 
and science curricula, MyTeachingPartner-Math/Science ( MTP-M/S ), 
and an accompanying teacher support system on children’s early 
mathematics and science learning in a sample of preschool chil¬ 
dren who are at-risk for negative school outcomes. To begin, we 
provide an overview of the previous research in this area and 
present the research-based model that supports the design of the 
MTP-M/S curricula and teacher supports. Then, we present our find¬ 
ings and discuss implications for the design and implementation of 
mathematics and science curricula and companion teacher support 
system aimed at improving children’s mathematics and science 
knowledge and skills. 

Informal development of early mathematics and science skills 

Over the past few decades, research has shown that, prior to 
any formal schooling, young children from ages 0 to 5 develop 


early informal everyday mathematics skills that are surprisingly 
broad and complex (Ginsburg, Lee, & Boyd, 2008) and at times, 
sophisticated (Zur & Gelman, 2004). This informal development 
typically includes ideas involving basic number sense and opera¬ 
tions (Baroody, Lai, & Mix, 2006; Bryant, 1995; Clements & Sarama, 
2007a), counting (Baroody, 1992; Frye, Braisby, Lowe, Maroudas, & 
Nicholls, 1989; Gelman &Gallistel, 1978; Stock, Desoete, & Roeyers, 
2009; Wynn, 1990) and geometric thinking (e.g., size, shape, loca¬ 
tion, and patterns; see Clements, 2004b; Clements, Swaminathan, 
Hannibal, & Sarama, 1999). Young children also begin to develop 
basic problem solving and an understating of simple calculation 
concepts (Levine, Jordan, & Huttenlocher, 1992). This informal 
mathematics development is not only a natural progression, but 
a fundamentally important life skill. As Ginsburg, Lee, et al. (2008) 
suggest, “everyday mathematics is so fundamental and pervasive a 
feature of the child’s cognition that it is hard to see how children 
could function without it” (p. 3). 

Research also indicates that young children can understand sci¬ 
entific concepts such as the life cycle, growth and change, and 
distinctions between animate and inanimate objects (Backscheider, 
Shatz, & Gelman, 1993; Inagaki & Hatano, 1996; Springer & Keil, 
1991). Moreover, preschool age children are capable of reasoning 
scientifically. For instance, they are able to infer how misleading 
evidence can lead to forming a false belief (Ruffman, Olson, Ash, 
& Keen, 1993). More recent research has suggested that by the 
age of six, children can differentiate between hypotheses and evi¬ 
dence (Ruffman, Perner, Olson, & Doherty, 1993; Sodian, Zaitchik, 
& Carey, 1991 ), which is earlier than prior research had suggested 
(Kuhn, 1989; Piaget & Inhelder, 1969). Thus, young children are not 
only capable of engaging in mathematics and science thinking and 
learning, but they also possess substantial informal understand¬ 
ings that can serve as the basis for formal mathematics and science 
knowledge and skills. 


MyTeachingPartner-Math and Science curricula and teacher 
support system 

MyTeachingPartner-Math/Science (MTP-M/S) curricula were 
designed in response to the need for high-quality pre-k mathemat¬ 
ics and science curricula. Their design was responsive to the foci 
above, informed by the research in early mathematics (Clements, 
2004a; Ginsburg, Lee, et al„ 2008; Klibanoff, Levine, Huttenlocher, 
Vasilyeva, & Hedges, 2006; Sarama & Clements, 2003) and science 
education (Duschl et al., 2007; French, 2004; Gelman & Brenneman, 
2004; National Research Council [NRC], 2006), and in alignment 
with national and state standards. The curricula possess similar 
activity designs and forms of teacher supports and also reference 
concepts reflective of both mathematics and science wherever 
possible, but were designed to also stand alone. Each curriculum 
includes two activities (each activity 15-20 min in length) every 
week, for 33 weeks across the school year. Weekly “center time” 
options enable the teacher to revisit specific mathematics and 
science activities with small numbers of purposefully selected 
students. Table 1 provides an overview of the curricular activity 
design, the activity domains and sub-domains addressed, and the 
numbers of activities corresponding to each. 

Table 1 also describes the MTP-M/S Teacher Support System 
to encourage teachers’ curricular implementation fidelity - both 
their adherence to the curricula as designed and the quality of 
their related interactions with students in the classroom - as well 
as support the development of teachers’ pedagogical and content 
knowledge. Some of these supports are embedded in the curricula 
and provided to all teachers (the “within activity supports” are 
part of the MTP-M/S Basic curricular package), and many others 
are separately delivered as part of MTP-M/S Plus (curricula plus 
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Table 1 

MyTeachingPartner-Mathematics/Science curricula and professional development intervention. 

For each curriculum (the MTP-M/S Basic curricular package): 

• 66 activities presented in explicit four-step inquiry format, each 15-20 min in length, in either whole or small-group format. Two activities are completed 

each week, for 33 weeks across the school year 

• "Within Activity” curricular supports (provided to all teachers using the curricula): 

- Identification of language to model and elicit 

- Recommendations for teachers’ questioning during the inquiry process 

- Adaptations to enable differentiated instruction for students needing more support or more challenge 

- A range of suggested extension activities 

• Weekly Centers: Choice of either mathematics- or science-related center activity 

• Mathematics- and Science-related books (18 for mathematics and 28 for science) and activity manipulatives 


Standards addressed 

Curricular domains 

Number of 
activities' 1 

MTP-M/S Plus teacher support system 

MyTeachingPartner-Mathematics 

• Number 

38 (57.6%) 

Online-only teaching supports provided include: 

Designed to address: 

- Oral counting 

16(24.2%) 

• Supports for every activity: 

• National Council of Teachers of Mathematics 

- Object counting 

28 (42.4%) 

- 2-3 min video demonstration 

(NCTM) Focal Areas for Pre-K (2006) 

- Numeral recognition 

22 (33.3%) 

- Brief teaching tip emphasizing one of the following: 

• Clements (2004) Developmental trajectories 

(inch foundational place 


O Best teaching practice 

for grades P-2 

value) 


O Concept knowledge 


• Operations 

11 (16.7%) 

O How children learn 


- Equal partitioning 

7(10.6%) 



- Combining/separating 

5 (7.6%) 

• Weekly 5 min challenge (n = 33): 


• Geometry 

10(15.2%) 

- Reflective prompt on teaching practice, to be answered while 


- Shapes 

8(12.1%) 

watching a provided videotape of one of the week’s activities. 


- Patterns 

3 (4.6%) 

Feedback on the practice issue provided by an expert in mathematics 


• Measurement 

13 (19.7%) 

or science education 


- Length 

6(9.1%) 



- Weight 

3 (4.6%) 

• Quality teaching dimension of the month: 


- Area/volume 

4(6.1%) 

- 1-2 min video demonstrates excellent practice on one of 10 

MyTeachingPartner-Science 

• Life science 

36 (54.5%) 

dimensions of quality 

State Standards used to refine focus for Pre-K, 

- Humans 

20(30.3%) 

- Brief description of how that dimension matters 

based on: 

- Animals 

20 (30.3%) 

- Links to library of over 150 video examples 

• K-2 Standards Benchmarks from the 

- Plants 

13 (19.7%) 


American Association for the Advancement of 

• Earth science 

16(24.2%) 

Professional devel. workshops: 

Science (AAAS)(1993) 

- Weather 

7(10.6%) 

• 1 full-day, 7 half-day (2.5 h) workshops 

• K-4 National Science Education Standards 

- Day/night 

5 (7.8%) 

- Intro to/review of quality teaching (2) 

(National Research Council, 2006) 

- Earth materials 

7(10.6%) 

- Mathematics and quality teaching (2) 


• Physical science 

20 (30.3%) 

- Science and quality teaching (4) 


- Properties of materials 

17(25.8%) 



- Movement 

9(13.6%) 



- Physical change 

10(15.2%) 



a Number and percentage out of the total of 66 mathematics or 66 science activities. Single activities frequently addressed more than one topical area within a domain, 
and occasionally more than one curricular domain. 


a teacher support system), which will be described later in this 
paper. 


MTP-M/S curricular design 

The activity designs for both mathematics and science reflect 
a focus on a careful scaffolding for knowledge and skill develop¬ 
ment across the year. As recommended byjustice and Pullen (2003 ), 
we attempted to balance meaningful student-centered interac¬ 
tions on topics that are relevant to children and teacher-guided, 
intentional teaching that ensures exposure to key skills and con¬ 
cepts. The activity design draws upon instructional theories related 
to structured inquiry, situated cognition anchored in authentic 
investigation, and cognitive development. Structured inquiry can 
be thought of as mid-way on an inquiry continuum that ranges 
from completely teacher-directed to totally student-driven (Rezba, 
Auldridge, & Rhea, 1999; as cited in Bell, Smetana, & Binns, 2005). 
In structured inquiry, teachers ensure exposure to key constructs 
by posing questions, suggesting procedures, and guiding students 
learning. At the same time, students are purposively invited to 
inquire: the teacher’s interesting questions engage their atten¬ 
tions as they work with the teacher and their peers to investigate 
(manipulating materials whenever possible) and observe related 
phenomena. The process concludes with students analyzing and 
discussing their observations, then extending and applying what 
they learned, a modification of the 5E Model (Bybee et al., 2006). 


Situating cognition, by anchoring instructional activities within 
the authentic experience of the everyday world, and the problems 
found within it, is thought to afford the development of richer 
understandings due to learners’ more active engagement - their 
situated cognition - while addressing the problems they tackle 
(Brown, Collins, & Duguid, 1989; Cognition and Technology Group 
at Vanderbilt [CTGV], 1990). In the context of MTP-M/S activities, 
students follow life cycles of plants, animals, and themselves across 
the year, both within and outside of their classroom. They make 
predictions and follow with observations and experimentation to 
answer questions about the natural and man-made worlds. For 
example, students come to understand what the plants around 
them need in order to thrive, by conducting an experiment: With 
seedlings they have planted, they make predictions and observe 
what will occur when they provide light and water, light but no 
water, water but no light, and neither light nor water. 

To develop mathematical understandings and skills, children 
count and measure themselves and their belongings, share num¬ 
bers of toys and snacks fairly, and recognize geometries in the world 
around them. These undertakings frequently begin with an authen¬ 
tic everyday problem, such as how to share cookies fairly with 
friends, for which relevant mathematics understandings (in this 
case, equal partitioning and combining/separating) are recalled, 
further developed, and applied, as when students share six cookies 
in different ways, to provide for two children, three children, or six 
children. 





MB. Kinzie et al. / Early Childhood Research Quarterly 29 (2014) 586-599 


589 


To help scaffold for both student and teacher conceptual under¬ 
standings, inquiries are often launched though use of children’s 
literature (18 books in mathematics and 28 in science). Through 
reading and discussion, the teacher and students develop a solid 
foundation from which to launch an inquiry, as in the case of an 
interactive reading of Bugs Are Insects (Rockwell, 2001), followed 
by students’ use of hand lenses for close examination of insects 
brought into the classroom and observed on nature walks. Book 
use preceding science inquiry has been found to be effective in the 
primary grades (Hapgood, Magnusson, & Palincsar, 2004). 

As part of our iterative curricular development/evaluation 
process, we discovered the importance of scaffolding teachers’ 
open-ended questioning and explicit language-development activ¬ 
ities, to stimulate students’ cognitive development. The importance 
of young children being able to think and talk about what they can 
do in both mathematics and science has been identified by leading 
researchers in the field (Gelman & Brenneman, 2004; Ginsburg, Lee, 
et al., 2008), as has the relationship between teachers’ mathemat¬ 
ical talk and students’ mathematical development (Klibanoff et al., 
2006). We incorporated many open-ended questions for teachers’ 
use, along with mathematics and science language to both model 
for and elicit from students. 

MTP-Math 

We drew upon the National Council of Teachers of Mathematics 
(NCTM) Focal Areas for prekindergarten (2006) and Clements’ 
(2004a) developmental trajectories for grades Pre-K-2, to artic¬ 
ulate our learning objectives for mathematics. The mathematics 
domains covered include; number sense, operations, geometry, and 
measurement. The “big ideas” - the most important concepts and 
skills - are addressed within each domain. Drawing upon the sub¬ 
stantial research conducted in this area, we place a priority within 
MTP-Math on development of children’s number sense (over one- 
half of MTP-Math activities address knowledge and skills in this 
domain), defined as involving written numeral recognition, oral 
counting (including organizing numbers in sequence), and object 
counting (extended to between-group comparisons of magnitude 
and subitizing) (Jordan, Kaplan, Nabors Ola’h, & Locuniak, 2006; 
Siegler, 1991). Early number sense is a prerequisite to learning 
place value, number composition and decomposition, basic arith¬ 
metic operations, and understanding of mathematical properties 
(Baroody, 2009; Griffin, 2004; Jordan, 2007; Miura, Okamota, Kim, 
Steere, & Fayol, 1993; NCTM, 2008; NMAP, 2008; Van de Walle, 
2003). Within MTP-Math, we reinforce early place value concep¬ 
tions using a number chart displaying color-coded numerals and 
corresponding ten-frames (blue for the tens place, orange for the 
ones place) to illustrate each number from 0 to 39. In addition, num¬ 
ber sense has been found to be an important predictor of academic 
success in early elementary grades (Jordan, Kaplan, Locuniak, & 
Ramineni, 2007; Stock et al., 2009) as well as in high school (Duncan 
et al„ 2007; Ginsburg & Allardice, 1984). 

Building on this strong foundation of number sense, students 
learn to perform simple mathematical operations, including equal 
partitioning and combining/separating of groups of objects. Recog¬ 
nition and creation of shapes and solids, as well as repeating 
patterns is the focus of geometry. Within measurement, students 
explore length, weight, and area/volume. 

MTP-Science 

The Benchmarks for Science Literacy from the American Associ¬ 
ation for the Advancement of Science (AAAS) (1993) and National 
Science Education Standards (NRC, 2006) articulate trajectories 
for grades K-2 and K-4, respectively; we used a review of state 
prekindergarten standards to refine our learning objectives for 
pre-k. The MTP- Science curriculum addresses three domains of 
science: life science, earth science, and physical science with 


inquiry-based activities to meet instructional objectives that are 
aligned with state and national standards. The activities guide stu¬ 
dents in construction of conceptual understandings across these 
broad domains, including (Life Science) Humans, Animals, and 
Plants; (Earth Science) Weather, Day/Night, and Earth Materials; 
and (Physical Science) Properties of Materials, Movement, and 
Physical Change. As shown in Table 1, our distribution of activities 
across these domains is in general alignment with that reflected in 
an assessment of national standards for pre-kindergarten science 
reported by Greenfield (2010). Activities were designed to provide 
opportunities to apply inquiry skills to predict, observe, analyze, 
and describe their findings, and to use simple science tools includ¬ 
ing a hand lens and balance. (For additional information on the 
MTP-M/S curricular design and its evolution, please see Kinzie, Vick 
Whittaker, McGuire, Lee, & Kilday, 2014.) 

MTP teacher support system 

As we previously describe, high fidelity curricular implemen¬ 
tation can be a struggle for early childhood teachers (Pianta et al., 
2005), and a particular problem where mathematics and science 
are concerned. This is due in part to limited professional develop¬ 
ment for teachers (NRC, 2006) and lack of teacher experience and 
confidence (Copley, 2004; Stipek, 2008). Teachers’ practice clearly 
matters for students’ learning; In the Tennessee state-wide Project 
STAR, between 12% and 14% of student performance in the pri¬ 
mary grades was accounted for by between-teacher variation in 
effectiveness, with effects much more pronounced for mathemat¬ 
ics achievement as compared to reading, and for lower-SES schools 
(Nye, Konstantopoulos, & Hedges, 2004). This relationship between 
lower-performing teachers and low-SES schools was underlined by 
results obtained by LoCasale-Crouch et al. (2007) in an eleven-state 
study of 692 pre-k classrooms: Teachers serving the highest pro¬ 
portion of children at-risk for early school failure were also those 
teachers performing at the lowest quality levels. Given the persis¬ 
tence of differences observed for children from low SES households 
(NCES, 2009, 2011), it seems especially important that their tea¬ 
chers receive the best possible support to implement high quality 
curricula effectively. 

Design of MTP-M/S teacher supports 

In acknowledging the inadequate preparation of the preschool 
teaching force, the National Research Council (2005) advises that 
curricula be comprehensive enough to enable success for teachers 
without strong preparation or experience. In response and to help 
encourage transfer to teachers’ practice, we articulated a range of 
teaching supports that would be quick and easy for teachers to 
use as they prepared to facilitate a learning activity. While some of 
these supports are “within-activity” supports provided to all MTP- 
M/S teachers (see Table 1 for a description), most are provided via 
the hybrid MTP-M/S Plus package tested here, including a combina¬ 
tion of on-line implementation supports, enhanced with a one-day 
summer workshop day and seven, 2.5 h teacher workshops (a total 
of 23.5 h outside the classroom). This hybrid format was intended 
to provide teachers with easy access to supports in multiple media, 
largely on the teachers’ own schedule, and minimizing the amount 
of professional development time required to effectively support 
teachers (compare to the average of 49 h of professional devel¬ 
opment found to be effective by Yoon, Duncan, Lee, Scarloss, & 
Shapley, 2007 in their review of effective programs). 

As with our curricular designs, a theoretical basis for design of 
the MTP-M/S teacher supports is informed by the tenets of situ¬ 
ated cognition, by anchoring the supports to teachers’ authentic 
tasks and encouraging their reflection on everyday practice (Brown 
et al., 1989; CTGV, 1990); we embed teacher supports in the phys¬ 
ical and social contexts of teachers’ practice (Putnam & Borko, 
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2000). The MTP-M/S professional development supports were also 
intended to promote high fidelity (including adherence and quality) 
implementation of the MTP-M/S curricula. Given that large-scale 
observational studies show clear linkages between the quality 
of teacher-student interactions and student learning outcomes 
(NCEDL and SWEEP, see Early et al„ 2005), we specifically focused 
on the characteristics of high-quality teacher-student interactions 
using a well-validated framework for encouraging classroom qual¬ 
ity, the Classroom Assessment Scoring System (CLASS; Pianta, La Paro, 
& Hamre, 2008). We also aimed to enhance teachers’ mathemat¬ 
ics and science concept knowledge and understandings of how 
to foster mathematics and science learning in early education, as 
recommended by the National Research Council (2006, 2009). The 
pedagogical and content focus for our teacher supports is summa¬ 
rized in Table 1, as well as the formats used, which are described 
next. 

MTP-M/S plus teacher support formats 

Drawing upon actual pre-kindergarten teachers’ implementa¬ 
tions of the curricula, we provided over 130, 2-3 min demonstra¬ 
tion videos that illustrated high-quality teacher-child interactions 
and highly inherent implementations of the curricula - one or 
more demonstration videos for every activity. Targeted video 
demonstrations have been found to be more effective than tex¬ 
tual descriptions in developing teachers pedagogical knowledge 
(Moreno & Ortegano-Layne, 2008). Video-based quality teaching 
challenges were also provided once each week, featuring one 
of the activities for the week and posing a reflective question 
on professional practice, to be answered while watching a short 
video clip of another teachers’ implementation. A mathematics 
or science educational expert offers feedback on that practice. 
As we have described elsewhere (Kinzie, Vick Whittaker, Kilday, 
& Williford, 2012), video-based observation and reflection can 
encourage authentic experiential learning (Kolb, 1984) and reflec¬ 
tion on action (Schon, 1987) without the pressures of being in “the 
teaching moment” (Borko, Jacobs, Eiteljorg, & Pittman, 2008). 

Videos were also used to illustrate a different dimension of qual¬ 
ity teaching (CLASS video and dimension of the month), including 
a 1-2-min video, demonstrating excellent teaching practice on the 
current month’s dimension of quality (e.g., quality of feedback), a 
brief description of the dimension and why it’s important, and links 
to more information on all ten dimensions of quality teaching, as 
well as a quality teaching library offering 150 video examples across 
many instructional settings, formats, and content areas. 

Brief teaching tips (we aimed for 25 words or fewer) were 
provided to highlight best pedagogical practices, common ways 
students construct understandings (including misconceptions to be 
on the alert for), or explanations of key mathematics or science con¬ 
cepts to aid teachers in their instruction. The Plus workshops were 
largely designed to help encourage teachers’ use of the on-line sup¬ 
port system. To help encourage their use of the on-line supports and 
provide opportunities for teacher reflection and peer discussion on 
their teaching practice, teachers in the MTP-M/S Plus group also 
participated in a one-day summer workshop and seven half-day 
(2.5 instructional hours each) workshops, spaced across the year. 
Each workshop focused on a specific dimension of quality teach¬ 
ing as defined in the Classroom Assessment Scoring System (CLASS 
Pre-K; Pianta et al., 2008), most in the context of mathematics activ¬ 
ities (n = 2 workshops) or science activities (n = 4 workshops), along 
with explorations of concept knowledge, self- and peer-reviews 
of teaching, error analysis activities (modeling common student 
behaviors as teachers practice identifying errors and guiding learn¬ 
ing, based in part of recommendations from the National Research 
Council (2006). 

In sum, while there is emerging research lending support for a 
number of mathematics and science curricula (Clements & Sarama, 


2008; French, 2004; Starkey et al., 2004), these curricular pack¬ 
ages are not yet in widespread use, and there are few products that 
offer both mathematics as well as science curricula. The goal of 
the present study was to fill this gap in the research by examining 
the effects of empirically based mathematics and science curricula 
and accompanying hybrid teacher support system on children’s 
mathematics and science knowledge and skills. 

Present study 

We conducted a small experimental study in pre-k classrooms 
during the 2009-2010 school-year to evaluate the potential of 
MTP-M/S to increase preschool children’s mathematics and sci¬ 
ence knowledge and skills across the pre-k year. Specifically, the 
purpose of the present study was to compare the development 
of mathematics and science skills for children whose teachers 
were assigned to one of three conditions: MTP-M/S curricula plus 
teacher support system (Plus), MTP-M/S curricula only (Basic), or 
Business as Usual (BaU). We hypothesized that students whose 
teachers participated in the MTP-M/S Plus and Basic conditions 
would show greater mathematics and science achievement in the 
spring compared with students whose teachers participated in 
the BaU condition. Because of the research suggesting the need 
for both high-quality curricula and support for their implemen¬ 
tation, we hypothesized that students whose teachers participated 
in the MTP-M/S Plus condition would return the best performance 
on assessments of mathematics and science skills, compared to 
students whose teachers participated in the Basic and BaU condi¬ 
tions. 

Method 

Participants 

The sample for the current study included 42 pre-kindergarten 
teachers and 444 of their students, from full-day state-funded class¬ 
rooms in a single school district of a large mid-Atlantic city (11 
Business as Usual teachers, 17 MTP-M/S Basic teachers, and 14 MTP- 
M/S Plus teachers). All teachers in this study taught in Title 1 schools 
which serve high percentages of children from low-income fam¬ 
ilies. The teachers were mostly female (98%) and ranged in age 
from 24 years to 65 years (M = 45, Median = 48, SD = 10.72). Teachers 
reported their race/ethnicity as Caucasian (54%), African American 
(44%), or Other Race (2%). Teachers held an average of 18.5 years of 
experience working professionally with infants to elementary-aged 
children (Median = 16.5, SD= 10.05) and all held at least a Bache¬ 
lor's degree (54% had a degree in early childhood education) as was 
a requirement for teachers in state-funded classrooms (see Table 2 
for descriptive information on teachers and classrooms). 

A total of 444 students participated in the study (an average of 
10 randomly selected students per classroom). Participating stu¬ 
dents were all kindergarten-eligible for the subsequent academic 
year, and average 4.76 years old (SD = .31). Parents of students (49% 
male) reported their race/ethnicity as African American (66%), Cau¬ 
casian (26%) or other race (8%). The sample was predominantly low 
income; the average income-to-needs ratio (computed by taking 
the family income, exclusive of federal aid, and dividing this by the 
federal poverty threshold for that family) was 1.34 (SD =. 98) with 
40% of households having ratios lower than one (below the poverty 
line) and 78% of families having ratios lower than two. There was a 
range in mothers’ highest level of education from eighth grade or 
less to a Master’s degree (high school diploma or less, 35%; some 
college but no degree, 28%; two-year degree or training certificate, 
28%; BA or above, 9%). Students’ demographic characteristics are 
presented in Table 2. 
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Table 2 

Child, family and teacher characteristics, by condition. 
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Child/family characteristics 

Control (business as usual) 
n = 116 

Curricula only (basic) 
n = 182 


Curricula plus supports (plus) 
n = 146 

(%) 

n 

Missing 

m 

n 

Missing 

(%) 

n 

Missing 

Gender 


114 

2 


181 

1 


140 

6 

Boy 

57% 



48% 



45% 



Girl 

43% 



52% 



55% 



Race/ethnicity 


115 

1 


182 

0 


140 

6 

African American 

60% 



71% 



64% 



Caucasian 

31% 



21% 



28% 



Other 

8% 



8% 



8% 



Maternal education 


101 

15 


175 

7 


139 

7 

Less than high school 

27% 



41% 



33% 



Some college but no degree 

34% 



26% 



25% 



Two-year degree or training certificate 

29% 



25% 



31% 



Bachelor’s or above 

10% 



8% 



11% 



Teacher characteristics 

Control (Business as usual) 

Curricula only (basic) 


Curricula plus supports (plus) 


n = 11 



n = 17 



n = 14 




(%) M(SD) 

n 

Missing 

(%) IW(SD) 

n 

Missing 

(%) M(SD) 

n 

Missing 

Teacher education 


9 

2 


14 

3 


13 

1 

Bachelor’s 

54% 



21% 



22% 



Bachelor’s plus one 

15% 



7% 



33% 



Master’s 

31% 



72% 



45% 



Teacher yrs experience 

14.41 (10.18) 



18.00(9.87) 



22.50(9.31) 




There was attrition of both teachers/classrooms and students 
over the course of the study. We conducted several analyses to 
examine potential attrition bias at both levels. Over the course 
of the study, seven teachers/classrooms dropped out: The school 
district pulled four teachers (two Basic teachers, two Plus) to par¬ 
ticipate in another study, while three teachers dropped for other 
reasons (personal circumstance [n = 1 Plus teacher], and work load 
[ n = 2 Business as Usual teachers]), for an attrition rate of 17%. To esti¬ 
mate attrition bias, we conducted analyses comparing teacher and 
classroom characteristics for the 35 teachers who fully participated 
and the seven teachers who dropped from the study. There were no 
significant differences (all p’s > .05) between the teachers who fully 
participated and those who dropped from the study on the covari¬ 
ates (mother’s education, family income, child gender, child race, 
teacher education, teacher years of experience) or the values of the 
pre-test measures assessed at baseline (TEMA-3, GMA, US, EPS). 

Initially, 416 students were selected to participate and com¬ 
pleted fall assessments. Of these, 69 students could not be assessed 
in the spring due to teacher withdrawal from the study or student 
withdrawal from the preschool. To offset child attrition in class¬ 
rooms where teachers were still participating, an additional 28 
students were selected from the original pool of consented students 
for the spring assessment. All students with some child data were 
included in the study (see description of analyses, below). There 
were no significant differences when comparing children who had 
both fall and spring data (n = 317) to children who had only fall 
or only spring data (n =127) with regard to age, gender, ethnicity, 
maternal education or family income. 

Intervention and previous findings 

In this study, we tested the effects of the MyTeachingPartner- 
Math/Science ( MTP-M/S) curricula and teacher support system 
on children’s math and science skills. We have reported else¬ 
where (Kinzie et al., 2012) on the fidelity with which teachers 
implemented the curricula, with this sample. As part of their par¬ 
ticipation in this study, we asked teachers to videotape themselves 
implementing MTP-M/S curricular activities. Tapes were coded for 
adherence to the curricular design. Overall, teachers in both Basic 
and Plus intervention groups showed fairly high adherence to the 


curricula as assessed by the MTP-M/S Fidelity Measure (Kinzie et al., 
2012). Across activities, out of a possible total of 24 points, Basic 
teachers scored an average of 13.67 (SD = 3.98) and Plus teachers 
scored an average of 15.19 ( SD= 1.97). In examining the distribu¬ 
tion on each fidelity item, we found that the majority of teachers 
got scores of “yes” on the dichotomous items and scores of “most” 
or “all” on the ordinal items. There were no significant differences 
in Basic and Plus teachers’ adherence to the curricula, however, in 
examining the standard deviations for each group, it appeared that 
there was more variability in teachers’ scores in the Basic group 
than in the Plus group. 

Finally, for teachers in the Plus group, we examined descrip¬ 
tive statistics regarding their use of implementation supports 
(Kinzie et al., 2012). Almost all of the teachers in the Plus group 
attended the initial summer workshop (with one day spent on the 
curricula) and the seven, 2.5-h professional development work¬ 
shops (M = 6.18 workshops/teacher, SD= 1.17). Teachers’ use of 
web-based resources was tracked using a server that automatically 
recorded the information about teachers’ access of the online sup¬ 
ports. There was a very large range in teachers’ website usage, with 
teachers ranging from 1.18 to 21.05h (M= 10.52, SD = 6.80) spent 
on the website. 

Materials 

Business as usual (BaU) group 

The district followed the HighScope curriculum (HighScope 
Educational Research Foundation, 2012) as a base, with a district- 
prepared pre-kindergarten curricular guide addressing a range of 
topics in oral language, literacy, mathematics, science, history, 
and social science. The district’s mathematics and science activ¬ 
ities were explicitly informed by state pre-kindergarten learning 
standards and by the High Scope curriculum, as well as by assess¬ 
ments of early learning (including the High Scope Child Observation 
Record [COR] and quarterly assessments in each subject area). 
Relevant to mathematics and science, the instructional strands 
addressed include: number and number sense; computation; mea¬ 
surement; geometry; data collection and statistics; patterns and 
relationships; scientific investigation, reasoning, and logic; matter 
(physical motion and forms of water); earth and space systems; 
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earth patterns, cycles, and change; resources; and history/change 
over time. 

Basic and plus treatment groups 

The Basic group received the MTP-M/S curricula described above 
and the teaching materials needed to implement the activities. The 
Plus group received the MTP-M/S curricula, needed materials and 
also access to the teacher support system of on-line professional 
development supports and related workshops. Both intervention 
groups were encouraged to fully implement the MTP-M/S curricula 
as well as any other mathematics and science activities they would 
like. 

Procedures 

Recruitment and random assignment 

Teachers were recruited from a single district in a large mid- 
Atlantic city. Invitation letters that described the mathematics and 
science curricula and the professional development supports were 
sent to all pre-kindergarten teachers in the district. Several infor¬ 
mational meetings were held with interested teachers to describe 
the study in more detail. Follow-up phone calls and/or in person 
meetings were held with interested teachers. 

Teachers who consented to participate attended an introduc¬ 
tory workshop in the fall, during which they were oriented to the 
purpose of the study, trained on the intervention to which they 
were assigned, and given information about data collection require¬ 
ments. Teachers in all three groups completed a survey describing 
their own and their classroom’s characteristics, in the fall and in 
the spring. 

The experimental evaluation was carried out using stratified 
random assignment with schools being randomly assigned to one 
of the three conditions. We stratified schools (n = 24) by number of 
participating teachers. Random assignment was conducted at the 
school level to try to prevent contamination of intervention effects 
across conditions. 

At the beginning of the school year, participating teachers sent 
home a consent form and short family demographic survey to all 
parents or guardians of their students, with a request that these 
be completed and returned. Ninety-four percent of parents con¬ 
sented to allow their children to participate in the study (529 out of 
578 parents). Based on the parental consent received, we randomly 
selected ten students per classroom for participation in fall and 
spring direct assessments. Students were excluded from the study 
if they were reported by their teachers to have an Individualized 
Education Plan for a severe developmental delay or learning disor¬ 
der (6% of students) or limited English proficiency (3% of students) 
because we did not have valid and reliable measures of students’ 
math and science knowledge and skills for these populations. 

Training for data collection 

Data collectors completed two full days of didactic training on 
administration of the direct child assessments (prior to fall assess¬ 
ments and prior to assessments in the spring). Data collectors were 
assessed using an extensive checklist during live practice to ensure 
that each data collector’s test administration and scoring skills were 
reliable and that they adhered to the standardized administra¬ 
tion manuals. Additionally, data collectors conducted four practice 
assessments on their own, and sent their data to the research team 
to be checked for completeness and accuracy, prior to assessment 
administration in the field. 

Child assessment protocol 

Data collectors were blind to the experimental condition of the 
students. At each assessment time point, students were brought 
to a quiet, private area and administered an assessment battery 


lasting a total of approximately 1 h (two sessions of 30 min each, 
with a 30 min break in between). After completion of the assess¬ 
ments, students were given a book for their participation. After 
all participating students had been assessed, all other students in 
these classrooms were given a book. 

The fall assessments were conducted approximately six weeks 
after the start of the school year, to enable teachers to establish 
classroom routines and provide a period of acclimatization for stu¬ 
dents who were new to the school experience. The ordering of 
children to participate in the fall assessment was random. Approx¬ 
imately 26 weeks (JVf = 26.47, SD = . 79) after completion of the fall 
assessments, the spring assessments were undertaken, with stu¬ 
dents assessed in the same order as in the fall, helping to ensure 
that approximately the same amount of time had elapsed for all 
students assessed. 

Measures 

Teacher and classroom characteristics 

On the fall survey, teachers reported about their professional 
experience (e.g., education level, field of study, years of experience) 
and their classroom composition (e.g., number of students, gender, 
ethnicity, and language). 

Child and family characteristics 

Parents or caregivers completed a survey that provided infor¬ 
mation about their child’s date of birth, gender, race/ethnicity, and 
Individualized Educational Plan (IEP) status, as well as maternal 
education, family income, and what languages were spoken in the 
home. 

Number sense and operations 

Students’ knowledge of numbers and numerical operations 
were tested using the Test of Early Mathematic Ability - 3rd Edition 
(TEMA-3; Ginsburg & Baroody, 2003). This standardized measure 
uses pictures and counting chips to assess students’ skills in number 
knowledge, such as cardinality, ordinality, one-to-one correspon¬ 
dence and enumeration, and their abilities in numerical operations. 
The TEMA-3 has parallel forms (A and B) and is designed to be given 
to children between the ages of 3 and 8 years, as either a diag¬ 
nostic tool for children having difficulty in a specific mathematics 
domain or to determine how a child is performing in relation to 
his or her peers. The measure is norm-referenced, and has been 
found to be a reliable and valid test of early mathematical abil¬ 
ity (Bliss, 2006; Ginsburg & Baroody, 2003). Concurrent validity 
has been reported by TEMA developers, with both the KeyMath R 
Basic Concepts subtest (r= .54) and the Young Children’s Achievement 
Test Math Quotient (r=. 91). In our administrations, this measure 
showed excellent internal reliability in both the fall (a = .91) and 
spring (a = .93). 

Geometry and measurement 

The Geometry and Measurement Assessment (GMA ) is a deriva¬ 
tive of the Tools for Early Assessment of Mathematics (TEAM ; Sarama, 
Clements, & Wolfe, 2011), and is specifically designed to assess chil¬ 
dren’s knowledge of shapes, patterns, measurement and positional 
words. For our study, six TEAM items were retained intact, seven 
questions were developed as extensions on questions in the TEAM 
(for example, with the TEAM, children are asked to make a triangle 
and a rectangle using coffee stirrers and we added a related ques¬ 
tion for making a square), and seventeen new items were developed 
to address additional related curricular objectives not assessed in 
the TEAM (for example, after a TEAM item asking children to iden¬ 
tify the longer of two sets of linking cubes, children are given a third 
set of linking cubes and asked to order the three sets from shortest 
to longest). Construct validity has been established for the TEAM 
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(Clements, Sarama, & Liu, 2008). In our administration, internal 
reliability for this measure was .82 in the fall and .86 in the spring. 

Number sense and place value 

In order to examine the effects of the MTP-Math activities focus¬ 
ing on foundational place value conceptions, and given the lack of 
existing measures focusing on this knowledge/skill for our pre-k 
population, project staff found it necessary to develop The Num¬ 
ber Sense and Place Value Assessment (NPV ). Dynamic assessments 
provide an indication of the level of support needed for perfor¬ 
mance (Pena, Iglesias, & Lidz, 2001), in contrast to conventional 
assessments that generally indicate only a static indication of per¬ 
formance on any given item (Justice & Ezell, 1999), and so provide 
more information about children’s level of understanding. 

The NPV contains two parts: Number Sense items evaluate stu¬ 
dents’ rational counting skills and numeral recognition skills, while 
Place Value items measure students’ concrete understanding of 
place value, as reflected by students’ ability to match numerals to 
corresponding 10-frame representations, specifically testing stu¬ 
dents’ emerging understanding of the ones place and tens place. 
Overall, the assessment includes nine items, with dynamic assess¬ 
ment implemented on seven of the nine items via scaffolding: 
When a child cannot respond to an item correctly, a series of scaf¬ 
folds (successive verbal prompts, some with use of 10-frame or 
number line manipulatives, with the final scaffold being joint per¬ 
formance with the adult) are provided, with each correspondingly 
reducing the possible score. Students can score a total of 41 points 
on the measure if no scaffolds are required to answer these items 
correctly. 

Prior to its use in the current study, the NPV was pilot-tested 
with 44 randomly selected students between the ages of 54 and 66 
months, drawn from a convenience sample of six pre-k classrooms 
intended to serve a low-income population in the mid-Atlantic. We 
observed that provision of scaffolds in this assessment improved 
children’s performance on both parts of the assessment, effectively 
identifying children’s level of successful performance before the 
final scaffolds available were required. Children’s performance on 
the two NPV components, number sense and place value, was sig¬ 
nificantly correlated (r=.511; p<.002; Kinzie et al„ 2014). 

With this administration we determined internal consistency 
for the NPV measure to be good, at .89. Concurrent validity was also 
determined with this administration: NPV performance was signif¬ 
icantly correlated with children’s performance on the mathematics 
achievement measures (TEMA, ft = 0.74, GMA, ji = 0.61, p < .01). 

Life sciences and earth and physical sciences 

As no science assessments were currently available for pre-k 
administration, the MyTeachingPartner-Math/Science research team 
created two assessments to enable examination of science learning 
outcomes. The Life Science Assessment (LiS ) and Earth and Physical 
Science Assessment (EPS) were created following a review of national 
and state standards for science learning in pre-kindergarten. A 
core set of objectives were identified through a process of cross- 
referencing between sets of standards. These objectives were used 
to create assessment items, in forced-choice and card-sort forms, 
as the National Science Education Standards (NRC, 2006) suggest 
that concept knowledge be assessed in multiple different ways. For 
forced-choice items, students are asked to point to the appropriate 
picture or material. For example, a student is shown a picture of a 
tree and is asked the question, “What is this?” before being asked 
the forced-choice question, “Are trees plants or animals?” For card 
sorting items, students are asked to sort photographs according to 
a specific (forced-choice) dimension (e.g., plants versus animals, 
items that will float if placed in water versus those that will not). 

The LiS assessment tests children’s understanding of the bio¬ 
logical world. These topics include living versus non-living things, 


characteristics of plants and animals, human and animal bodies, 
using the senses, and plant and animal life cycles. The assessment 
contains 73 forced-choice and card-sort items, each worth 1 point. 
The measure had good internal reliability in the fall (a = .83) and 
adequate internal reliability in the spring (a = .77). 

The EPS assessment determines students’ understandings of sci¬ 
entific tools, weather, temperature, material composition, motion, 
and buoyancy. The assessment contains 19 forced-choice and card- 
sort items with a total possible score of 39. The measure had 
adequate internal reliability in the fall and spring (a = .74 and .77, 
respectively). 

Multi-level analyses 

The structure of the data included three potential levels at which 
the data could be analyzed: child, classroom, and school. Because 
randomization occurred at the school level, it could be argued that 
this is where the clustering should occur. However, the number 
of schools included (N= 24) is too small to trust multilevel models 
that are performed using this variable as a cluster variable (Maas & 
Hox, 2005). We performed interclass correlations for both schools 
and teachers for all of our models and found that the ICCs were 
very similar. Therefore, we chose to fit two-level HLM models that 
accounted for the nesting of students within classrooms. The result¬ 
ing data structure involved an average of 10 children in each of the 
42 teacher’s classes. Data were analyzed using Mplus version 6.1 
(Muthen & Muthen, 1997-201 0). Missing data for any one variable 
ranged between 6% and 20%. Analyses were run using full informa¬ 
tion maximum likelihood estimation so that data analyses used all 
available data from each case (444 students across 42 classrooms) 
when estimating parameters and therefore increasing the statis¬ 
tical power of estimated parameters (Enders & Bandalos, 2001). 
Demographic variables that have typically been included as control 
variables in the developmental literature were included: chil¬ 
dren’s ethnicity, child gender, and maternal education (Moilanen, 
Shaw, Dishion, Gardner, & Wilson, 2010; Smith, Calkins, Keane, 
Anastopoulos, & Shelton, 2004; Wanless, McClelland, Tominey, & 
Acock, 2011 ). 

We fit a series of models examining differences in children’s 
mathematics and science skills for children in the Business as Usual, 
Basic, and Plus groups, controlling for fall mathematics and science 
skills and child characteristics. The level-1 model specifies that chil¬ 
dren’s spring mathematics or science skills are a function of their 
fall pretest score, gender (girl = 1, boy = 0), ethnicity, and mater¬ 
nal education (continuous variable ranging from eighth grade or 
less to a doctorate degree). Three dummy variables were created 
for ethnicity and entered simultaneously so that Caucasian eth¬ 
nicity served as the reference group and the different indicators 
corresponded to African Americans, Hispanics, and those of other 
ethnicities. 

In the level-2 model, teachers’ study condition was entered. Two 
dummy variables were created and entered simultaneously so that 
Business-as-Usual teachers were the reference group and the dif¬ 
ferent indicators corresponded to those in the Basic group and those 
in the Plus group. The level-2 model also included the classroom- 
level aggregate of the pretest score, the teacher’s education level, 
and the teacher’s total amount of teaching experience as covari¬ 
ates. We calculated effect sizes for the significant treatment effects 
using Hedge’s g, dividing the difference between adjusted means 
for the two groups by the standard deviation for the outcome in the 
control group. 

Results 

Table 3 provides descriptive statistics for outcome variables 
included in our models. We conducted Wald Z omnibus tests to 
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Table 3 

Pre and post-test scores for outcome measures, by condition. 


Outcome measures 

Control ( business as usual ) 

N = 116 


Curricula only {basic) 

N= 182 


Curricula plus supports (plus) 
N= 146 



Mean (SD) 

n 

Missing 

Mean (SD) 

n 

Missing 

Mean (SD) 

n 

Missing 

TEMA-3 

Pre 

11.94(6.73) 

109 

7 

9.50(6.55) 

168 

14 

10.87(5.91) 

138 

8 

Post 

19.38(9.14) 

87 

59 

16.2(8.87) 

145 

37 

18.41 (8.28) 

107 

39 

GMA 

Pre 

13.35 (4.96) 

109 

7 

11.11 (4.71) 

168 

14 

12.25(4.34) 

138 

8 

Post 

16.49 (4.93) 

87 

59 

15.15(5.62) 

145 

37 

17.91 (4.28) 

107 

39 

NPV 

Pre 

Post 

29.93 (6.16) 

87 

59 

28.79 (7.16) 

145 

37 

31.48(5.76) 

107 

39 

LiS 

Pre 

38.68 (5.76) 

108 

8 

38.89(5.24) 

168 

14 

38.48 (5.56) 

138 

8 

Post 

43.28 (4.35) 

87 

59 

42.72 (4.58) 

145 

37 

44.22 (4.56) 

107 

39 

EPS 

Pre 

37.24 (5.94) 

107 

9 

35.16(6.18) 

168 

14 

35.71 (5.97) 

138 

8 

Post 

40.42 (6.29) 

86 

60 

39.34(6.39) 

144 

38 

40.59(5.83) 

107 

39 


examine whether there were baseline differences across groups 
( MTP-Plus , MTP-Basic, Bail) with regards to teacher (i.e„ highest 
level of education, years of experience) and classroom character¬ 
istics (i.e„ average maternal education), and children’s pre-test 
scores. We observed a significant group differences in teachers’ 
level of education (Wald Z= 6.46, p = ,04), such that those in the 
Basic group had the greatest education, followed by those in the 
Plus group, followed by those in the BaU group. We found no 
significant differences across groups on any of the other teacher 
or classroom characteristics. There were significant differences 
in children’s mean scores on pre-test assessments of children’s 
number sense and operations skills (Wald Z=8.29, p =. 02), where 
those in the BaU group had the highest pre-test means (M = 11.94, 
SD = 6.73), followed by those in the Plus group (M = 10.87, SD = 5.91), 
followed by those in the Basic group (M = 9.50, SD = 6.55). There 
were also significant differences in pre-test assessments of chil¬ 
dren's geometry and measurement skills (WaldZ= 10.28, p = .006), 
where those in the BaU group had the highest scores (M= 13.35, 
SD = 4.96), followed by those in the Plus group (M = 12.25,4.34), fol¬ 
lowed by those in the Basic group (M= 11.11, SD = 4.71). Baseline 
differences in the other child outcomes were not significant. 


Are there differences, based on treatment condition, in children’s 
mathematics gains across the pre-k year? 

Unconditional models suggested that between 3% and 19% of the 
variance in children’s mathematics scores was between classrooms 
(lCCs = .03-.19), with the remainder attributable to child charac¬ 
teristics. Coefficients for the covariates included in our models are 
provided in Table 4. The estimated marginal means for the different 
treatment groups (controlling for the covariates) are presented in 
Table 5. Comparisons for the homogenous subsets comparing the 
BaU group to the Basic and Plus groups are based on the tests of 
the dummy codes from the multilevel model discussed above. The 
comparison of the Basic to the Plus group is taken from a dummy 
code in a second model that was identical to the model discussed 
above except that the Basic group was the reference group for the 
treatment dummy codes. 

There was a significant omnibus effect of treatment on chil¬ 
dren's geometry and measurement skills (Wald 2=6.32, p = .04). 
Specifically, controlling for students’ fall scores and demographic 
characteristics, those in the Plus group had significantly greater 
gains on the GMA than students in the BaU group (effect size g = .52). 
Students in the Basic group did not differ from those in the Plus or 
BaU groups. 


There was also a significant omnibus effect of treatment on chil¬ 
dren’s number sense and place value skills (Wald Z=6.88, p = . 03). 
Students in the Plus group scored significantly higher on the NPV 
measure compared to students in the Basic (g = .35) and Business as 
Usual conditions (g= .47). Students in the Basic group did not differ 
from those in the BaU group. 

There was not an overall effect of treatment on children's gains 
in number sense and operations as measured by the TEMA-3 (Wald 
Z= .07, p = .97). 

Are there differences, based on treatment condition, in children’s 
science knowledge and skills at the end of the year? 

Unconditional models suggested 4-7% of the variance 
in children’s science scores was attributable to classrooms 
(lCCs = .04-.07). There was no overall effect of treatment on 
children's gains in life science ('Wald Z=3.77, p =. 15) or earth and 
physical science (WaldZ= 1.30,p = .52). 

Discussion 

This study examined the potential of MyTeachingPartner- 
Math/Science for improving preschool children’s mathematics and 
science knowledge and skills. We hypothesized that students 
whose teachers participated in the MTP-M/S Plus and Basic con¬ 
ditions would show greater mathematics and science achievement 
in the spring compared with students whose teachers participated 
in the BaU condition. Further, we hypothesized that there would be 
greater achievement for students of teachers in the Plus group com¬ 
pared with the Basic group. These hypotheses for positive treatment 
effects for the Plus group over the Basic and BaU groups were par¬ 
tially supported for the mathematics domain, but not for science. 

In comparison with the BaU group, students of Plus group tea¬ 
chers demonstrated (with medium effect size) greater learning in 
geometry and measurement. Geometry and measurement are two 
areas that are fundamental concepts in cognitive development, 
but which typically receive less attention in pre-k classrooms 
(Clements & Sarama, 2011 ). In our study, compared with Business- 
as-Usual classrooms, MTP-M/S classrooms had a greater emphasis 
on geometry and measurement (15.2% of MTP-Math curricular 
activities focused on geometry [compared to 12.6% in the BaU class¬ 
rooms] and 19.7% of MTP-Math activities emphasized measurement 
skills [compared to 15.2% of activities in the BaU classrooms). The 
learning trajectories in these areas are complex and require careful 
scaffolding by teachers to advance children’s skills, however, 
early childhood teachers have had few professional development 
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Table 4 

Effects of MTP-M/S intervention on children’s development of math and science skills during pre-k. 



TEMA-3 (number sense/ 
operations) 

Pi.SE) 

GMA (geometry/ 
measurement) 

P(SE) 

NPV (number sense/ 
place value) 

pm 

LiS (life sciences) 

P(SE) 

EPS (earth/physical 
sciences) 

P(SE) 

Level-1 Covariates 

Pre-test 

0.740 (0.034) 

0.496 (0.050) 


0.606 ” (0.039) 

0.499 (0.044) 

Gender (girl) 

-0.036(0.028) 

0.062(0.051) 

0.028 (0.052) 

0.026 (0.048) 

-0.026 (0.048) 

African American 

-0.011 (0.041) 

-0.166 (0.042) 

-0.105 (0.065) 

-0.065 (0.049) 

-0.172 (0.042) 

Hispanic 

0.025 (0.045) 

0.090(0.051) 

-0.015 (0.037) 

0.068 (0.05) 

-0.008 (0.058) 

Other 

0.06 (0.039) 

-0.029 (0.036) 

0.053 (0.056) 

-0.032 (0.047) 

-0.063 (0.054) 

Maternal education 

0.03 (0.039) 

0.094 (0.043) 

0.175 (0.066) 

0.01 (0.05) 

0.048 (0.049) 

Level-2 covariates 

Pre-test 

0.854(1.833) 

0.928 (0.244) 


0.791 (0.509) 

0.548 (0.464) 

Teacher education 

0.032 (0.997) 

0.004 (0.206) 

0.426 (0.236) 

-0.078(0.481) 

0.468 (0.574) 

Teacher experience 

0.106 (0.399) 

-0.149(0.190) 

-0.008 (0.233) 

-0.006 (0.267) 

-0.202(0.353) 

Treatment effects 
(Basic-control) dummy 

0.148(1.839) 

0.392 (0.332) 

-0.173 (0.266) 

-0.341 (0.27) 

-0.063 (0.354) 

(Plus-control) dummy 

-0.014(0.537) 

0.601 (0.258) 

0.485 (0.222) 

0.314(0.342) 

0.379(0.408) 

(Plus-basic) dummy 11 

-.163(1.347) 

0.234 (0.232) 

0.678 (0.225) 

0.669 (0.273) 

0.457 (0.450) 


a The effect of the (basic-plus) dummy code was taken from a second model using Basic as the reference group for the treatment dummy codes. 
’ p<.05. 

" p<.01. 

•" p<.001. 


experiences related to teaching geometry and measurement 
(National Research Council, 2009). The teacher supports for those 
in the MTP-M/S Plus group offered opportunities for teachers 
to view effective implementation of activities in these areas by 
viewing the demo videos, perhaps increasing the quality of their 
implementation of these activities (as compared to those in the 
MTP-Basic group). 

Students of teachers in the Plus group outperformed students in 
both the Basic and Bail conditions in number sense and place value 
skills (with small to medium effect sizes). This Plus group advantage 
may have been the result of these teachers’ participation in two 
monthly workshops focusing on best pedagogical practices related 
to number sense and place value, as well as access to substantial 
related online supports provided for all curricular domains (includ¬ 
ing Demonstration videos and Teaching Tips underlining pedagogy, 
ways children construct meaning, and key mathematics and sci¬ 
ence concepts). The online supports in the number sense and place 
value domains may have been particularly useful as MTP-Math 
activities included learning objectives that were developmentally 
more challenging than traditional pre-kindergarten lessons that 
cover basic number sense concepts such as oral counting and num¬ 
ber recognition. For example, MTP-Math activities cover topics such 
as subitizing, stable order-principle, cardinality, composing and 
decomposing numbers, and unitizing into groups of five and ten. 

With regard to children’s gains in the TEMA-3 assessment of 
knowledge and skills in numbers and operations, the marginal 
means in the treatment groups were greater than the marginal 
mean for the Bail group but the differences were not significant. 


The lack of between-group differences on this measure may have 
been due to the similarly strong weighting of number sense activ¬ 
ities in the MTP-Math curriculum (57% addressed knowledge and 
skills in this domain) as well in Bail classrooms, where we observed 
47.4% of mathematics activities implemented emphasizing number 
sense. It may also be that the TEMA-3 did not discriminate well at 
the point of the fall pretesting; as noted by Bliss (2006), this mea¬ 
sure does not have an appropriately deep floor for children until 
they are aged 4 years, 3 months. 

The results we obtained for geometry and measurement out¬ 
comes as well as for number sense and place value suggest that the 
combination of the MTP-M/S curricula and teacher support system 
offer some benefit for teachers serving children from low-income 
families. In these areas, it appears that the provision of profes¬ 
sional development supports were necessary to produce significant 
gains in children’s scores. Professional development is thought to 
be key to educational reform (Sarama, DiBiase, Clements, & Spitler, 
2004), with increases in teacher professional development and in- 
service education linked to improvements in classroom quality and 
children's development (Bowman et al., 2001). This may be espe¬ 
cially true in math domains such as geometry and measurement 
and number sense and place value. In these domains, teachers not 
only need to describe and demonstrate correct use of numbers 
but may also be called on more directly to apply mathematical 
knowledge for teaching. Pedagogical knowledge for mathemat¬ 
ics includes the ability to use pictures or diagrams to represent 
mathematic concepts to students, as well as analyze students’ con¬ 
ceptual development (Hill, Rowan, & Ball, 2005); both are called 


Table 5 

Estimated spring marginal means and homogenous subsets by condition. 



Business as 
usual (control) 

Curricula only 
(basic) 

Curricula plus 
supports (plus) 

Overall test for 
treatment (Wald Z) 

TEMA-3 (number sense and operations) 

17.66665 a 

17.81465a 

17.65265a 

Z=. 07 
p = . 97 

GMA (geometry and measurement) 

11.7188a 

12.1108 ab 

12.3198 bc 

Z=6.32 
p = .04 

NPV (number sense and place value) 

29.74892a 

29.68492a 

30.38592b 

Z=6.88 
p = . 03 

LIS (life science) 

43.37576 ab 

43.03476a 

43.68976 bc 

Z=3.77 
p = .15 

EPS (earth and physical sciences) 

39.91307a 

39.85007a 

40.29207a 

Z= 1.30 
p = . 52 


Note. Means within a row that do not share any subscripts are significantly different (p < .05). Means for TEMA-3, CMA, US, and EPS control for Fall scores and all covariates. 
Means for NPV only control for covariates because there was no Fall assessment of NPV. 
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upon to effectively scaffold students’ geometry and measurement 
and number sense skills. 

Other studies examining effects of mathematics curricula on 
students math skills have found a range in effect sizes from small 
(0.08; Sophian, 2004) to large (1.07; Clements & Sarama, 2008). Our 
moderate effect sizes for geometry and measurement (0.52) and 
number sense and place value (0.47) fall in the middle. Those inter¬ 
ventions that found large effect sizes (Clements & Sarama, 2007b, 
2008) tended to include a coach for teachers, and also extensive 
hours of professional development. For example, MTP-M/S teachers 
in the Plus group received 23.5 h of professional development across 
both mathematics and science curricula, compared to 50 h of pro¬ 
fessional development (including 16 h of personalized classroom 
observation and coaching twice each month) offered in support 
of the single Building Blocks mathematics curriculum (Clements & 
Sarama, 2008). 

There is an emerging group of studies suggesting that the posi¬ 
tive effects associated with interventions may be more likely seen 
in teachers’ second year of implementation, relative to their first. 
Clements, Sarama, Spider, Lange, and Wolfe (2011a) purposively 
delayed direct assessment of student learning until the second 
year of teachers’ implementation of Building Blocks, with positive 
results for this curriculum (ES = 0.72). In the Preschool Curriculum 
Evaluation Research (PCER, 2008) evaluation of Pre-K Mathemat¬ 
ics (supplemented by DEM Early Childhood Express Math Software ), 
80% of teachers were in their second year of curricular implemen¬ 
tation; beneficial results were found on an abbreviated form of 
the Child Math Assessment, (ES = 0.44) and on the Building Blocks 
Test of Shape Composition (ES = 0.96). In an evaluation of Big Math 
for Little Kids (BMLI<), an effect size of 0.43 was identified in favor 
of this intervention, but only when students had entered kinder¬ 
garten (during which time they continued to participate in BMLK 
curricular activities) (Ginsburg, Lewis, & Clements, 2008). These 
findings suggest the importance of an ongoing program of cur¬ 
ricular implementation to ensure teacher familiarity, and also 
continued engagement in learning experiences across the early 
childhood years. It could be that we would see stronger effects on 
children’s math outcomes, were teachers to implement MTP-M/S 
for a second year. 

We did not find any significant differences in the omnibus tests 
across groups in children’s gains in life science or earth and physical 
science. We note that, for LiS, students of teachers in the Plus group 
had significantly greater gains compared to those in the Basic group. 
For EPS, the marginal means for the Plus condition were greater 
compared to the BaU condition, although these differences were 
not significant. There have been only a few published results of 
science curricular trials for comparison. In these trials, researchers 
have found significant improvements in children’s skills in areas 
other than science. Results from implementation of the Early Child¬ 
hood Hands-On Science (ECHOS) curriculum suggest its benefit; with 
outcomes determined through teacher ratings using the Galileo 
measure. Results suggested that children in treatment classrooms, 
compared with control, made greater gains in Approaches to 
Learning, Early Math, Language and Literacy, and Creative Arts. 
However, there were not significant differences in children’s 
science skills (Greenfield, Jirout, et al., 2009). (In small trials of the 
ScienceStart! Curriculum, the child outcomes have been assessed 
with a measure of receptive vocabulary, rather than science knowl¬ 
edge and/or skills, with improvements of 0.5 standard deviation; 
French, 2004). These studies show that science interventions can 
have positive effects on children’s skills in domains other than 
science. In this study, we did not assess children’s skills in areas 
other than science and math. A lack of well-validated and sensitive 
measures in the area of science is also suggested by this research. 
Our project-developed science measures were developed by the 
research team over a relatively short time period; it may be that 


these measures need additional refinement in order to discrimi¬ 
nate well. As noted by Brenneman (2011), “science is not among 
the domains that are well represented in the catalog of reliable and 
valid assessments available to educators and researchers” (p. 2.) 

In this intervention, we were aiming to improve children’s 
skills in two domains, as compared with many curriculum-focused 
interventions that focus on a single domain (Clements & Sarama, 
2007b, 2008; Clements, Sarama, Spitler, et al„ 2011; French, 2004; 
Ginsburg, Lewis, et al., 2008; Greenfield, Jirout, et al., 2009). This 
decision was purposeful, as math and science are two areas that 
are critically important to children’s later learning (Claessens & 
Engel, 2013; Duncan et al., 2007; Grissmer et al., 2010; NMAP, 
2008), and are not well represented in teacher’s instruction (Early 
et al., 2010). However, asking teachers to learn new curricula in 
not just one, but two subject areas, had implications for their 
cognitive load as well as the dosage of activities that children 
received and the amount of professional development afforded in 
each domain. By way of comparison in the area of dosage, stu¬ 
dents in Building Bloclcs classrooms experience about twice the 
amount of whole- and small-group classroom mathematics activ¬ 
ities that MTP-Math/Science includes. In addition, Building Blocks 
students spend 10 minutes/week on related computer activities. 
These factors may be helpful in interpreting our effect sizes rel¬ 
ative to those obtained in the other curricular trials described 
above. As researchers move toward development and implementa¬ 
tion of cross-curricular packages addressing student learning and 
development in multiple domains, these moderate effect sizes may 
become the norm. 

Limitations 

There are several limitations that deserve attention. First, we 
conducted our analyses at the classroom level although random 
assignment was at the school level. One of the negative conse¬ 
quences that resulted from randomizing at the school level with 
a limited number of schools was that we saw some pre-test dif¬ 
ferences in children’s skills. Children in classrooms assigned to 
BaU displayed significantly higher mathematics skills (number 
sense and operations and geometry and measurement) in the fall. 
Although we controlled for children’s pre-test scores in related 
analyses, these differences suggest that BaU classrooms started the 
year with higher levels of math achievement than those in inter¬ 
vention classrooms. There is research to suggest that here may be 
a differential rate of growth in academic skills for students who 
those who start the year with higher versus lower skills, with chil¬ 
dren who start the year with higher skills growing at a higher rate 
compared with their peers (Downey, von Hippel, & Broh, 2004). It 
is possible that children in the BaU classrooms not only started 
the year with higher levels of math skills, but that these skills 
also grew at a faster rate than those in the Basic and Plus class¬ 
rooms. 

Second, unanticipated attrition at the classroom level (seven 
teachers/classrooms, 17% of sample, due to a district-level admin¬ 
istrative decision), with 71% of attrited classrooms having been 
assigned to a treatment condition, negatively impacted our power 
to detect significant differences between treatment groups. A pri¬ 
ori power analyses accounted for attrition at the child but not 
classroom level. Although we accounted for missing data in our 
analyses, this attrition resulted in lower power to detect signifi¬ 
cant results. And, because this attrition was not distributed across 
treatment condition it lowers our ability to make causal infer¬ 
ences. 

Third, the effect sizes for our intervention were smaller com¬ 
pared to some other curriculum and PD packages focusing on early 
math or early science skills (Clements & Sarama, 2008; Clements, 
Sarama, Spitler, et al., 2011; French, 2004). This is likely due to the 
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lower intensity of MTP-M/S, due in part to the fact that our curricu¬ 
lum activities and PD sessions were split across two content areas. 

Finally, given that several of the measures employed here were 
recently developed ( Number Sense and Place Value [NPV], Life Sci¬ 
ence [LiS], and Earth and Physical Science [EPS]), care must be taken 
with the corresponding results obtained here. As described previ¬ 
ously, these instruments were developed by the MTP research team 
in the absence of available measures for these domains of knowl¬ 
edge and skill, and working from the same learning objectives that 
guided curricular development. This is not an unusual practice in 
research in early childhood settings where few assessments exist in 
mathematics and science; Clements, Sarama and colleagues have 
expended substantial effort to develop their assessment of early 
mathematics learning (Clements & Sarama, 2007b, 2008; Clements, 
Sarama, Spitler, et al., 2011; Clements, Sarama, & Wolfe, 2011) as 
have Starkey, Klein, & Wakeley (2004), while Greenfield has been 
similarly engaged in development of direct assessments of science 
knowledge and skills (Greenfield, Dominguez, et al., 2009) while 
undertaking evaluative research on the ECHOS science curriculum. 
We note that the NPV was administered at post-test only. It is 
possible that different results would be obtained when variance 
associated with a pre-test is controlled for. Further study of these 
measures across early childhood settings, including situations in 
which our curricula are not in use, and in conjunction with other 
measurement will help us better understand the characteristics of 
these measures. 


Conclusion and directions for future research 

The findings reported here suggest that MTP-M/S curricula and 
teacher supports have value for encouraging mathematics learning 
for young children potentially at-risk for early school failure, adding 
to the emerging research in this area (Clements & Sarama, 2008; 
French, 2004; Starkey et al., 2004). We found in other analyses 
(Kinzie et al., 2012) that in the study described here, MTP-M/S was 
implemented with a high degree of adherence in both Basic and Plus 
classrooms, suggesting that the curricular instructions are accessi¬ 
ble to teachers. Further, we found that the MTP-M/S Plus teachers 
had strong attendance at the professional development workshops 
and used the online teacher supports provided, which led to signif¬ 
icantly less variability in teachers’ adherence when compared with 
Basic teachers. 

In our current research, we are evaluating the effects of the MTP- 
M/S curricula and teacher supports over multiple years, enabling 
teachers to surmount any “learning curves” involved in learning 
to implement these new curricula. In other studies, assessment of 
effects was deferred until teachers’ second year of implementa¬ 
tion, to good effect (Clements, Sarama, Spitler, et al., 2011; PCER, 
2008). We are also collecting data on children’s skills in domains 
other than mathematics and science, as some interventions that 
include math and/or science curricula have found impacts on 
language (French, 2004; Greenfield, Jirout, et al., 2009; Sarama, 
Lange, Clements, & Wolfe, 2012) and executive function (Weiland 
& Yoshikawa, 2013). 

We are beginning to develop a better understanding of how 
to effectively support teachers’ instructional interactions, to assist 
them in building a solid foundation for young children's mathemat¬ 
ics and science learning. It is our hope that through research such 
as this, we can identify mechanisms (e.g., high-quality curricula 
and teacher professional development) through which we can help 
develop children’s knowledge and skills in both mathematics and 
science. Ultimately, we hope that through these efforts, we will 
identify how to best support teachers in implementing high-quality 
mathematics and science curricula, and positively impact children’s 
learning outcomes in these domains. 
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